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DEVICES WITH WINDOW - TIME - SPACE ^OW CONTROL (WTFC) 
DESCRIPTION OF THE INVENTION 

1. FIELD OF THE INVENTION 

This invention concerns devices with data flow control, used in packet switched telecommunications 
networks. According the international classification (MKP) it is classified as: H 04 L 12/56 - 
SECTION H - ELECTRICITY, TRANSMISSION OF DIGITAL INFORMATION, Data 
switching networks, packet switching systems. 

2. TECHNICAL PROBLEM 

Modem telecommunications networks use advances of information statistical multiplexing, achieved 
with packet switching, despite packet length is variable up to some limit (Internet) or fixed (ATM), 
and despite packets are forwarded individually or using virtual channels. Users segment their data to 
packets in terminal equipment and send them to the network, while network nodes route packets to 
their destinations. 

In case when, in the time of observation, a network node receives less packets than the capacity of 
output channels used to forward those packets, the networic is underutilized. In case when more 
packets are received, some of them will be transferred up to the full capacity of output channels, and 
the rest will be buffered in node's memory. The network is overloaded, and congestion has occurred. 
Because of packet buffering, the average packet must wait for transmission of the previous packets, 
and its waiting time is prolonged. 

Statistical properties of incoming traffic (packet network terminals send packets at arbitrary 
moments) influence occasional congestion and network underutilization. To provide quality of 
service (QoS) it is necessary to minimize the ntunber of packets in node's queues, but to the level that 
will provide satisfactory network utilization. This is optimal network working point, kept by 
congestion control measures, including congestion avoidance when congestion is not apparent, and 
congestion elimination, after for some reason congestion has occurred. The most important 
congestion avoidance measure is flow control. 
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m flow control in packet networks is achieved by packet sending rate regulation in network 
terminals. In case of network underutilization, terminals should increase, otherwise, in case of 
congestion, diey should decrease packet sending rate. 

3. PRESENT STATE OF THE TECHNIQUE 

Th. flow control problem is apparent for last ten years, and a number of investigators work on it 
worldwide. Despite numerous scientific papers and granted patents, the very feet, that old 
unsatisfectory solutions from 1988. are used in the Internet nowadays, shows that right solution of 
the flow control problem has not been discovered yet. 

The basic mechanism of congestion occurrence is packet buffering in node memory. Packets are 
stored to queues, waiting to be transferred through chamiels. Prolonged packet waiting time is a 
consequence of packet bufifcring. Hie time packet spends on it's route from source to destination, and 
similar time acknowledgment spends in backward direction, are together called Round Trip Time 
(RTT, or simply T). and its prolongation because of waiting we consider to be basic quality of 
service measure in packet switching networks. 

The flow control solution is searched using two approaches. In the first approach, network nodes do 
not provide data about network state, but rather store excessive packets in memories, or discard 
remaining packets when memories are fiill. Terminals of such neUvorks measure round trip time T, 
window W (the number of their own packets on the network) and packet losses, and try to adjust 
their packet sending rate. In the second approach, network nodes provide data about network state 
implicitly (e.g. enforcing waiting time or packet loss rate) or explicitly (e.g. sending forward or 
backward congestion indications, or even signaling the optimal sending rate for the data flow). When 
explicit notification is used, the data processing load in nodes can be very large, prohibiting the use 
of some recently proposed solutions. In all cases, except when optimal rate is explicitly signaled, 
packet transmitter defines new optimal rate using embedded transmitter algorithms (linear increase 
and exponential decrease of rate are most frequently used). 

4. THE INVENTION ESSENCE EXPOSITION 

-nie invention in this patent request deals with the solution of the congestion in packet networks by 
proposing improved flow control method. The novelty of the invention, related to the present state of 
the technique, is flow control in window - time space, used to achieve optimal packet sending rate 
and unifoma times between packets. This provides high network utilization and high quality of 
service, while keeping network node processing load at minimal level. 
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Devices with window - time - space flow control (WTFC) use network model based on the 
determined single server queuing system and finite number of packets in the network. The total 
network capacity point (WoTo) defines window - time plane (or space), which contains networic 
response curves for various fractions of corresponding capacity. Each response curve has a break 
point, which lie on the hyperbola througli (Wo.To). After a point (W,T) is measured, it is possible to 
determine whether it is positioned above or below the hyperbola, and apply one of formula sets to 
calculate corresponding capacity fraction and break point coordinates of the response curves. 
Coordinates of the break point are used to calculate optimal window and delay, and their ratio to 
obtain optimal packet sending rate. 

Nodes can signal the total network capacity (Wo,To) in form of elementary network channel 
parameters. Using the bottleneck concept, device with WTFC control obtains (Wo,To) point 
coordinates. Alternatively, (Wo,To) can be estimated from previous measurements. 

4-1, The Black Box Model of the Network 

We observe data communications network as a black box, meaning that network structure is not 
known to the packet sender (transmitter). Packet sender sends packets to the network and receives 
acknowledgments. It observes round trip time as an difference between time of the acknowledgment 
and time of packet emitting. Equation 1 : 

T = t{at)-t(p,) 1 

In the same time, the window is measured as the difference, at the moment of packet emitting, 
between packet sequential number and last acknowledgment sequential number. Equation 2: 

W = pu-aj 2 

The receiver has a role of packet destination and acknowledgment generation. 

We model a network using single server queuing system. Figure 3. Regarding the characteristics of 
telecommunications channel. Equation 3 is valid: 

T = Tq+To=Tq+T,+Tp ^ 
To=T,+Tp 

where T, total waiting time of the model, is round trip time, Tq is queue waiting time and To is total 
serving time, equals the sum of propagation time Tp (depending on channel length and propagation 
speed) and transmission time T, (a time needed to send packet to the channel, depending on packet 
length and channel capacity). Considering at most one packet in the server, for a single server system 
we have Equation 4: 
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W = W, +Wo=W, +W,+Wp ^ 
Wo=W, +Wp = l+Wp 

where W is window (total number of packets), W, is number of packets in the queue and Wo is total 
number of packets currently served, equals the sum of packets in the server W.=I.and number of 
packets in the channel Wp. For packet switching networks Equation 5 is valid: 

T=wM = M=l 

* C w.=i C n 5 

where C is channel capacity or transmission speed (bits per second), M is average length of packets 
(bits per packet) and is serving rate (packets per second). We also use X to mark packet arrival 
rate (packets per second). 

4.2. Optimal Working Point of the Packet Switching Network 

Optimal working point of the packet switching network is detennined as a compromise between 
quality of service (delay) and network utilization, kept to provide economical operation of the 
network. If a network is modeled using the single server queuing system. Figure 3. we use response 
(delay) curves, i.e. delay dependence on network load. M/M/1 model widi markov (exponential), 
G/G/1 model with general, and D/D/1 model with deterministic probability dense functions of packet 
interarrival and service times, are on our disposal. Deterministic model is a special case of general 
one when variances are equal to zero. If we sketch approximate delay curves, it is obvious that 
lowering the variance the G/G/1 curves smoothly transform to D/D/1 one. Figure 4. The M/M/1 case 
can be considered the worst one because of relatively high variance. 

Optimal working point can be determined using the maximum network power (ratio between 
throughput and delay) or the quality of service criteria. Using maximum power criteria. Figure 4, 
lowering the variance network utilization raises slowly, but with simultaneous increase of quality of 
service. Using constant quality of service criteria. Figure 4, lowering the variance networic utilizaUon 
raises fest. Delay curve close to the D/D/1 one is obtained by lowering the variance for only one 
order of magnitude. 

One can conclude diat flow control mechanism, besides adapting the packet sending rate, should 
provide low variances of packet interarrival times and packet service times, hi that case, high 
network utilization is possible while keeping quality of service constant, and the flow control 
mechanism can use delay curves close to D/D/l one. 
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4.3. The Window - Time Space 

Delay curves on Figure 4 are sketched for the case of infinite possible number of packets in d»e 
network, and all of them have infinite value when network load approaches 100%. in praxis, we can 
limit the number of packets a transmitter is allowed to -end to die network. That number is called 
maximum window. IF we apply the finite window concept (also called preservation of number of 
packets in the network) to the deterministic queuing system, we get a D/D/i/W system with delay 
curve according Figure 5. The delay fimction gives a curve with broken in the (WoJo) point. Wo is 
optimal window that makes fiiU network utilization possible, and consists of one packet in die server 
an Wo-1 packets in the channel. The curve is defined with Equation 6: 

' To W^Wo 
T _ J T» 6 

. Wo 



Lowering the window, delay time remains constant and equal to To=Tp+T., i.e. the sum of the 
Pi propagation and servicing times. Equation 3. The network is undenitilized. Increasing the window 

packets are stored to the queue and linear delay increase, according to the line through origin, occurs. 
Network is overloaded, but delay time does not raise to infinity because number of packets is limited 
(finite). 

Using Equation 6 and delay curve from Figure 5, we define window-time space, i.e. W,T plane. 
Each point of the plane represents a (W,T) pair of window and delay time. If measured (W.T) point 
is positioned on the curve. Figure 6, the communications network is presented exactly with this 
D/D/l/W system. Otherwise, when measured dot is out of the curve, die network parameters have 
been changed, and new working point should be obtained. 

4.4. Network working point estimation 

If user knows the coonlinates of the (Wo,To) point, and consequently the corresponding delay curve, 
and if he measures the (W.T) point positioned on the delay curve, he can assume that the total 
network capacity is on his disposal, and that his optimal working point is exactiy (W0.T0). In praxis, 
user shares network edacity with others. We assume that all of them use a littie, but equal (fair) 
portion of the total capacity. Equation 7: 

Hj = a^ ; a e(0,l) 7 

where a=l/n is a fector dependent on number of users. The sloped part of the curve is defined widi 
Equation 8: 
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and consequently, Equation 9: 



1 To (a 

T = w-i- or T = W-^ 
an Wo(a) 



To(a) = Wo(a)— ; X = an 



Considering Tp is independent of a, we have Equation 10: 

and we can calculate delay for the flat part of the delay curve. Equation 1 1 : 

Tn (a) = Wo(a) — = (Wp{a) + l)— = Wp (a) — + — = Tp + — 



10 
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The delay curve is determined with Equation 12: 

1 

T = 



T„+ X^a\i 



ajt 

W— X>an 
afi 



or T = < 



To(a) W^Wo(a) 

W-3°ti W>Wo(a) 
Wo(a) ^ 
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Depending on parameter a we got a family of curves whose break points lie on the hyperbola like 
curve. Figure 6. 

From the Equation 11 it is obvious that new optimal delay time To{a) consists of constant 
propagation time and transmission (service) time which is increased for several times because other 
users utilize network capacity. Measuring a (W,T) point in the arbitrary point of the W,T plane, one 
from the femily of delay curves is chosen and explicit calculation of parameter a is enabled. We first 
check whether the measured (W,T) point is above die hyperbola using two criteria. Equation 13: 



T>To{a) 
W>Wo{a) 



13 



or using Equations 9 and 10: 



TST„ + 



an 



WsT_an + l 
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15 



Arranging and combining Equations 14 and 15 we get Equation 16: 
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Using the formula for Tp, Equation 3, we get Equation 17: 



T„ _ W„ - 1 



Tp = To - Tj = To - — = To - -J- - T, 



IZI 17 



Wo "° Wo 

that we use to obtain basic criteria for judging whether point (W,T) lies above or below the 
hyperbola. Equation 18: 

W-1 ^ To 18 

Wo-1 (T-To)Wo+To 

If condition in Equation 18 is fulfilled, the (W,T) point lies above the hyperbola, and parameter a is 
calculated using Equation 19: 

%i ap. a Wo Wo T 

p If condition in Equation 18 is not fulfilled, the (W,T) point lies below the hyperbola, and parameter 

= a is calculated using Equation 20: 

W T-T ra^-T + — ^ =5. a = — 20 

^ T-To(a)-Tp+^ =*a ^^(t-To) + To 

ru 

Obtained a is used to calculate optimal working point (Wo(a),To(a)) according to the Figure 7. If the 
(W,T) point lies above the hyperbola, the transmitter has overioaded the network and it should 
reduce its packet sending rate. If the (W,T) point lies below the hyperbola, network is underutilized 
and packet sending rate should be increased. The packet emitting period to(a) is calculated fi-om the 
optimal delay and window ratio. Equation 21: 



T„ + 



1 To 



, , To(o) . . ^ aWo _ 1 To 



21 



0 



If condition in Equation 18 is fiilfilled, using Equation 19 we get Equation 22: 

t /„u^-3°- = i 

If condition in Equation 18 is not fulfilled, using Equation 20 we get Equation 23: 



22 
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% Wo " Wo vvo 

Equations 22 and 23 are simple and suitable for efficient implementation of the transmitter sofhvare. 
Besides optimal packet sending period we also calculate optimal window to ensure finite number of 

packets in the network. Equation 24: 

^Vo(a)=To^cx^■*■l = {Wo-l)a-l " 

Wn In 



25 



If condition in Equation 18 is fulfilled, using Equation 19 we get Equation 25: 

WT W 

Wo(a) = (Wo-l)^-l = Tp--M 

If condition in Equation 18 is not fiilfilled, using Equation 20 we get Equation 26: 

n 11- = — ?^ + l 26 

Wo(a) = (Wo-l)^^^^_^^j^^^+l Wo(T-To)+To T-T, 

In praxis, formulas with Tp from Equations 25 and 26 should be used for their simplicity. 

4.5. Total network capacity estimation 

The total network capacity estimation is concerned with determination of tiie (Wo,To) point 
coordinates. In reality, network is very complex structure and transmitter, using the black box 
approach, under the total netNvork capacity in fact determines ti,e capacity of the patii used to 
transfer it's packets. In networks with virtual path (ATM, X.25) tiiis path is always tiie same for the 
given comiection. In networks %vithout virtiial channel (Internet) packets are forwarded individually, 
where it is observed that tiiey follow the same forward patii in great percentage, while 
acknowledgments follow different backward patii. 

The total capacity can be estimated using tiie measured parameters, or signaled from die network 
nodes. Estimation metiiod is less precise, but does not require network nodes processing It appears to 
be more suitable for networks without virtual cliannels. Signaling metiiod is more precise, but 
requires additional network nodes processing in tiie phase of connection establishment. It appears to 
be more suitable for networks using virtual channels. Alternatively, signaling metiiod can be used in 
networks witiiout virtual channel when ne^vork nodes signal total capacity for every packet 
forwarded. 
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With estimation method, measurements of round trip time T, window W and service time T.=l/n 
are available. Service rate ^ is limited with the slowest channel on the path, the bottleneck channel. If 
minimal round trip time T=To and serving time are measured. Equation 27 follows: 



Tp=To-T, and Wo=^ = ToH 



27 



The measurement of parameters Tp and T, can be difficult. We have supposed that minimal round 
trip time measured is equal to To. In real network diere is no guarantee that measured minimal round 
trip time even approaches real To. Similar difficulties are present with serving time T, measurement. 
The packet pair method gives acceptable results on simple networks only, because there is no way to 
avoid the influence of phase effects from other users. Packets from a pair can be spread apart after 
the bottleneck channel. 

With signaling method, we start from the assumption that packet forwarding path will not change, 
or that network with virtual channel is in question. Here we analyze channel parameters that one can 
obtain from the given node. 

The channel is uniquely determined widi its capacity Ci(b/s) and propagation delay Tp. Hie total 
propagation delay on the m-chamiel path is of interest for user, Equation 28: 

Tpo=ZTpi ; i = l...m 28 

i 

and that value fully corresponds to the propagation time of the single server model. 
Transmission (service) time is variable and in accordance to Equation 5 depends on channel capacity 
and mean length of the packet. Concerning that die same packets are forwarded along the padi, the 
same packet length distribution can be observed on all segments, resulting diat differences in 
transmission times depend only on individual channel capacities. Total service time on the path is. 
Equation 29: 

T,o=i:T,i ; i = l...m 29 

However, a path consists of serial connection of channels, widi optimal speed equal to the speed of 
the slowest channel. Here we use the bottleneck concept. Equation 30: 

T,b =m3x(T,i) ; i = l - m 30 

For die transmission time we use bottleneck serving time T.b according to Equation 30, while 
transmission times of other channels can simply be added to the propagation time. Equation 3 1 : 
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+ T.K 31 



To =ZToi = STpi +17,; =[ STpi + ZT.i J +T,b 

i i i 

Using Equation 3 we get Equation 32: 

Tp=ZTpi + ZT,i=Tpo + ZT,i and T, =T,b 32 

To achieve efificient operation of network nodes, it is optimal to signal elementary channel 
parameters, that can be used at the transmitter to calculate needed values. Nodes can signal total- 
propagation time by adding propagation times of their own channels to the value of appropriate 
cumulative variable in the packet header. Equation 33: 

Tpo=ZTpi 33 

Concerning the transmission time depends on packet length M and channel capacity, it is convenient 
for a node to signal the channel capacity. Using Equation 5 for a single server model, we have 
Equation 34: 

T,o=ZTsi=I^ = MZ-^ 34 

i i I 

Network nodes should accumulate reciprocal values of channel capacity to another cumulative 
variable in the packet header. According Equation 31, we also need the bottleneck service time, 
which can be easily obtained by signaling the bottleneck capacity. Equation 35: 

Cu -min(Ci) ; i = l-'-m 35 
i 

A node will signal the bottleneck channel capacity by inserting to the packet header variable the 
edacity of its ovm channel only if it is lower than current value of the variable. Transmitter, 
knowing the mean length M of it's own packets, calculates transmission (service) time Ts, 
Equation 36: 

and propagation time Tp, Equation 37: 

Tp =ZTpi +IT,i -T,, =Tpo +T,o -T, =Tpo + m(z^--^) 37 

Minimal propagation time To is calculated according to Equation 3, and window Wo according to 
Equation 27. 
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4.6. Total network capacity correction 

At the beginning of operation, i.e. at the or immediately after the logical channel establishment, 
transmitter estimates initial values for (Wo.To). Estimated parameters may not be correct. During the 
operation, transmitter can measure such a dot (W,T). that does not correspond to the actual W,T 
plane. Figure 8. 

If parameters (W0.T0) are estimated corrccUy, measured dot can not be located in the area defined 
with Equation 38: 



T< 



To W^Wo 

T„ 38 
W-i5- W>Wo 

Wo 



on Figure 8 under the delay curve, because To is minimal serving time and Wo is a window of 
packets needed to fill the chaimel and server. 

Suppose, instead, that (Wo,To) are incorrect in a way that real network capacity is greater than 
estimated. New network capacity point (W,; ,T^) will be positioned in the part of W,T plane below 
the delay curve on Figure 8. Two cases are possible. Equations 39 and 40: 



Tp <T< 



To W^Wo 

T 39 
y^Jo. W>Wo 

Wo 



^ T^T 40 



In the case of Equation 39, the (W^ ,T<;) point is located below the delay curve, but above the 
horizontal line Tp, Figure 8. In that case, three possibilities of using earlier measurements are 
available, to assume that Tp was correct, that T, was correct or that window Wo, i.e. ratio To/T., was 
correct. 

Tp correct assumption: We can assume that propagation time Tp is estimated correcUy, meaning 
that network (botdeneck) capacity is wrong. In the case of lower capacity, the efifect is the same as 
the regular corresponding capacity fluctuations due to change in number of other data flows. In the 
case of capacity increase, new working point (W^ ,T^) should be estimated. The assumed Tp 
correctness implies that path is not changed, and the difference between old and new value of optimal 
delay. To -Tq , can according to Equation 3 appear because the lower service time T, only. 

Here we use the property of curve Wo(a), Equation 1 1, that it extends to the area for a>l, or to the 
right in W.T plane where W>Wo, the feet that was not used before. According to Figure 8, the curve 
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is extended to the right of point Wo. Suppose a situation when, after using parameters (Wo,To), a 
new round trip time Tp<T<To is measured. We take this value as the new minimal time: 

To'=T 41 

It is necessary to determine new window Wo . We start with Equation 17 and get Equation 42: 

T T' 

T„=Tn 2_ =7,; 2_ 42 



resulting with Equation 43: 



T' T " ^ To-Wo(To-To') TJ-Tp 
*^ Wo 

Transmitter will constantly track difference between To and T, and when needed adjust total network 
capacity parameters using Equations 42 and 43. Assumption that earlier value of Tp was correct is 
very risky because of smooth hyperbola slope for a>l. If Tp parameter was incorrect, optimal 
window correction using Equation 43 gives unacceptably high values. 

T, correct assumption: If we assume that transmission time T, is correct, we have badly estimated 
Tp based on incorrect To. In that case we use new minimal value according to Equation 41, and 
modifying Equation 27 we get Equations 44 to calculate Tp and W©: 

Tp=To'-T, , W^=^ 44 

If correct T, assumption is only partly correct. Equations 44 give acceptable and stable results. New 
optimal window is lower than previous one. 

Wo=To/T, correct assumption: Here we assume that transmission and propagation times are 
evaluated with the same error. Based on the constant W© using new TJ and according to 
Equation 41 we calculate new service time. Equation 45: 

T' 

T, =-i2- 45 
Wo 

and propagation time using Equation 44. This method provides stable results, because it preserves 
old optimal window. It is somewhat more aggressive compared to constant T, method, which results 
with lower window, but is still for more stable than the correct Tp method. 
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In the case of Equation 40, the (W,; ,T,;) point is located below the horizontal line Tp, Figure 8. 
Because of T<Tp, parameter Tp is definitely not correct. We can suppose that major change in 
network operating mode happened, or that earlier estimation was completely wrong. As die 
consequence, we are aware xhM previous value of Wo. which should denote number of packets in the 
channel and in the server, in reality includes some packets in the waiting queue. Equation 46: 

Wo=W^+W,' 46 

However, because W^ is unknown, we cannot simply determine W^ . Here we can assume that such 
a drastic change in delay time has occurred vwthout service rate decrease. Equation 47: 

_L<1 => JLkJi. 47 
^' H Wo 



and consequently. Equation 48: 



W^^^Wo 48 

^0 



In the same time we have actual measurement of window W. If new, minimal delay was measured 
with window larger than the value of Equation 48, we take measured value for Wq . Transmitter will 
operate using Equations 41 and 49: 

Wc; =max|^|2.Wo , W j 49 



4.7. WTFC auxiliary algorithms 

From the transmitter point of view, we have three outside events that stimulate its activity: 

1. Data to be sent, delivered from communications user 

2. Acknowledgment, received from the network 

3. Interpacket delay timer expiration. 

We distinguish three phases in the flow control establishment: 

1. First and further packets sending, but before first acknowledgment is received 

2. After first acknowledgment, but before second one is received 

3. After second acknowledgment is received. 

Transmitter will send packet to the network primarily after the interpacket delay timer expiration. If 
optimal window has already been achieved at that window, packet sending is postponed until next 
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acknowledgment is received. Interpacket delay timer expiration we recognize as a credit, and only 
one credit can be memorized. Transmitter operation using optimal rate i/to(a) with one credit 
provides packet emitting without bursts, while usage of optimal window Wo(a) criteria preserves 
finite number of packets in the network. This two parameters bring the network to the optimal 
working point. 

Connection startup algorithm: In die first phase, transmitter has no knowledge about the networic 
capacity. This phase lasts until die first acknowledgment is received, i.e. equals (initial) round trip 
time T. Depending on network structure, distance and routing, and network load, round trip time can 
be anywhere between Ims and more than 10s, the ratio of values is 1:10000. In the same time, 
available bit rate can be in range from 15,000 to 15.000.000 b/s or more. 

Networks are characterized with bandwidth-delay product x+CT. When low transmission speed is in 
question, large delay is of no concern because small amount of data can be transferred during the 
single round trip time.. The situation is very similar with high speed low delay networks. In both 
cases, bandwidth-delay product is relatively low. Problems arise with high speed high delay 
networks, when bandwidth-delay product is high. With such networks, transmitter inactivity of 
several hundred milliseconds results with resource utilization loss of tenths of megabits. Parameter x 
exactly represents number of bits in channel at fiill utiUzation, but in the same time the number of 
not transferred bits in case of transmitter pausing during one round trip time. 

It often happens that high x network users need to transfer less than % amount of data. That means 
that these data could be sent before the first round trip time period passes, only if transmitter should 
have known his relevant network capacity. On the other send, sending larger number of packets to 
the network in the first phase is questionable because working point of the network is not known. 

Here we use an algorithm that enables sending of moderate number of packets in the first phase, and 
still not to congest the low capacity network. If total capacity estimation is used, algorithm sends 
packet pair at the very beginning of the data transfer. Packets pi and p2 are used to measure 
bottleneck capacity. If total capacity signaling is used, only on packet (pi) is sent at the beginning of 
operation. After that, transmitter continues to send packets periodically with exponential slowdown. 
The time between two subsequent packets k and k+1, tk, is of double duration compared the time 
before previous packets k-l and k. Equation 50: 

tk =102*^-* ; k^2 50 

where to is one half of the initial period. The graph of packet sending dynamics is presented on 
Figure 9. 
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In the moment when first acknowledgment is received, the second phase begins. In this phase we wait 
for reception of second acknowledgment and we send packets using the achieved sending period t^.u 
Figure 10. First acknowledgment reception can occur in the first half of the period tk=2tk.i, and we 
wait that half to expire to send the packet. Otherwise, if first acknowledgment is received in the 
second part of the period r^^, new packet is emitted immediately. In both cases, timer is initiated to t^. 

This way, long waiting for t^ expiration is avoided, and lower ambiguity about packet emitting 
moment after the first acknowledgment is achieved. Because of this algorithm, during the first phase 
timer is always initiated to half of the desired value, and initial packet sending period is 2to. The 
same timer circuit can be used for startup as well as regular packet sending period determination. 
Measured window correction: The flow control algorithm calculates optimal working point using 
the measured pair (W,T), i.e. window and round trip time. Here, window is number of packets or 
octets in the network just at the moment when the packet, whose acknowledgment has just arrived 
and provided new (W,T) pair measurement, was emitted. With pure window control networks, new 
packet is sent at the moment when acknowledgment of some eariier packet was received. The 
window is an integer value. Figure 1 la. 

With WTFC network packet is emitted at the moment of transmission period timer expiration, which 
is independent of acknowledgment reception (under condition that optimal window has not been filled 
yet). Considering that last acknowledgment was received before the current packet emission, we can 
assume that a part of packet on die network is already transferred. The integer window value is 
corrected taking round trip time into account. Figure lib. As we do not have exact information 
about the transferred data quantity, the linear approximation is used. Equation 5 1 : 

W. (I. 51 
C ^(A,)-.(Ai) 

where t(Ak) and t(Aj) are moments of observed packet acknowledgment reception and of last 
acknowledgment reception, before observed packet emission took place. t(Pk) is the moment of 
observed packet emission. For t(Pk)=t(Aj) Equation 5 1 is reduced to Wk=(k-J), and is identical to 
Equation 2. 

Corresponding capacity estimation algorithm: After second acknowledgment, a third phase of 
transmitter operation begins. We now have the total network capacity parameters (Wo,To), and we 
are able to establish flow control. In this phase, the most important event is acknowledgment 
reception. In that moment we have new (W,T) pair, suitable to calculate packet sending period to(a) 
and optimal window Wo(a). 
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^3 



□ 



In praxis, parameter to(a) according Equation 22 should be used with care. The actual measurements 
of round trip time and window (W,T) are in fact data from the past. Parameter to(a) should be used 
to obtain optimal window in the future. Therefore, in real WTFC device the optimal window Wo(a) 
is calculated first, and than working value to (a) using Equation 52: 



Wo{a) 



52 



Parameter filtering: Feedback system stability depends significantly upon delay. The network 
response to any transmitter action will be noticeable only after die round trip time. Therefore, two 
adaptive filters are used, depending on round trip time and window size: rate slowdown and rate 
period to (a) filtering. 

Rate slowdown is used because WTFC model, based on D/D/lAV system, actually approximates 
real G/G/i model. Real working point should be positioned to tlie left and above of the W,T plane 
curve break point. Packet sending rate slowdown is achieved using empirical relation. Equation 53: 

ti'(a) = toW(l+YTo) 53 

so when e.g. y=0,l and To=ls about 10% of rate reduction is achieved. With low round trip times, 
good system stability is expected, and operation near the full network utilization is possible. With 
longer delays, rate slowdown improves stability of operation. 

Period t'o{a) , achieved using Equation 53, is additionally filtered using exponential smoothing, 
nonlinear regarding the direction of value change. If new rate is lower tiian previous, filtering is 
skipped, and new, larger value of t{;(a) is effective immediately. When rate is increased, the 
adaptive filter dependent on window size is used. Equation 54: 

where factor p is forgetting factor dependent on window Wo(a), Equation 55: 

0 Wo(a)^a 
p^|Wo(°)-% a<Wo(a)<b 55 



b 

c b <Wo(a) 
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Changing the values of a and b, the area of linear increase of P with Wo{a) is changed. Changing the 
value of c, the slope of linear dependence is changed, and that is also the biggest filtering fector, 
achieved with large windows. Useful filtering parameters are e.g. a=0, b=10 and c=0,925. 

4.8, Devices with window-time space flow control 

Packet switched data communications networks with window-time-space flow control (WTFC) are 
constructed using two basic types of equipment, WTFC nodes and WTFC terminals. Nodes 
generally forward packets, and are often called routers. Terminals are devices that allow users to 
access the network, and most frequently are user computers. Disregard the actual device construction 
and purpose, the important issue is that nodes and terminals perform the WTFC algorithms. 

To implement WTFC mechanisms, besides the well known packet and acknowledgment numbers, 
packet number must contain additional variables according the Table 1. In packet header we have 
ip dual variables of each kind, to enable acknowledgment transfer inside the backward packets. 



03 


variable name: 


forward 


backward 


Q 


propagation time cumulative variable 


Tpof 


Tpob 




reciprocal capacit>' value cumulative variable 


Scif 


Scib 


s _ s 


reciprocal minimal channel capacity variable 


Cinunf 


Ciminb 


1 W 


last acknowledgment variable 








sending time variable 


T(pjc)f 


T(pk)b 



Table 1 . Packet header variables needed for WTFC implementation 



Terminals v^ith WTFC control include two essential modules, the transmitter and receiver of packets 
or other user data units processed with WTFC, Figure 1 . 

Packet transmitter (101) forms packets dividing user data in segmentation process (102), or when 
no user data are pending for transmission, forms acknowledgment packets after the packet from the 
opposite direction is received, based on the receiver request. 

Initialization process (103) writes appropriate values to the packet header variables. Forward last 
acknowledgment variable aif is set to the value of last acknowledgment number received, and 
backward last acknowledgment variable aib is set to the value of the same forward variable, received 
in the opposite direction packet whose acknowledgment number is contained in the current packet. 
Forward sending time variable T(pk)f is set to the value of current local time, and backward sending 
time variable T(pk)b is set to the value of the same forward variable, received in die opposite 
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direction packet whose acknowledgment number is contained in the current packet. If network is 
signaling total capacity, forward propagation time cumulative variable Tpor and forward reciprocal 
capacity value cumulative variable Scif are cleared, and reciprocal minimal channel capacity variable 
Ciminf is set to maximal value. In the same time, values from variables Tpor, Scif, i Cimnf received in 
the header of the opposite direction packet, are copied to the same backward variables of the current 
packet. Formed packets are stored in the packet buffer (104) and sent to the packet sending 
process (105) in accordance with credit in credit buffer (108) jfrom transmission rate to"'(ct) 
calculation process (107) and optimal window check (111) of Wo(a), if current window is lower 
than optimal. 

Wa(a) calculation process (110) uses Equations 25 and 26 to calculate it, and to'"(a) calculation 
process (107) works according to Equations 52 and 23, or 53 and 54, both using condition from 
Equation 18 from tlie area check process (112). 

After each acknowledgment reception, receiver signals to transmitter the acknowledgment number 
and time of acknowledgment, along with values stored in header variables listed in Table 1. Using 
these parameters, the (W,T) point measurement process (106) calculates particular values for T 
using Equation 1 an W using Equation 2, together with correction according the Equation 51. 

On every or some of acknowledgments received, depending on method applied, the network total 
capacity estimation and correction process (109) calculates parameters (Wo,To), and Tp using 
Equations 27 through 49. 

Packet receiver (113) uses it*s reception process (115) to receive packets from the channel and it's 
extraction process (114) to extract and deliver received data to the user, and needed header 
parameters and time of acknowledgment to the transmitter. In the same time, it sends an order to the 
transmitter to send acknowledgment included in the opposite direction packet header, or using 
separate acknowledgment packet if currently there are no user data to be sent. 

Nodes (201) forward packets using forwarding process (202), and in the same time, on the WTFC 
networks, can signal total capacity parameters if signaling method is used. Figure 2. In that case, 
capacity signaling process (203) modifies forwarded packet header, updating die forward 
propagation time cumulative variable Tpor using Equation 33, forward reciprocal capacity value 
cumulative variable Scif according Equation 34 and forward reciprocal minimal channel capacity 
variable Ciminf using Equation 35. 
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5, SHORT DESCRIPTION OF FIGURES 

Figures 1 - 11 are used in exposition to achieve easier invention essence description and 

understanding. Figures 1 and 2 present the structure of WTFC devices, and the rest of them are used 

to illustrate the operation of WTFC algorithms. 

Figure 1. WTFC terminal structure 

Figure 2. WTFC node structure 

Figure 3. Queuing system with one server 

Figure 4, Delay curves and optimal working point for M/M/1, G/G/1 and D/D/1 models 

Figure 5, D/D/1 AV model response 

Figure 6, Family of delay curves depending on a 

Figure 7. Optimal working point calculation 

Figure 8. Constrains in W,T plane 

Figure 9, Connection startup algorithm, with packet pair 

Figure 10. Packet sending algorithm after first acknowledgment 

Figure 11. Window calculation for a) window control and b) combined control 

6, DESCRIPTION OF INVENTION REALIZATION 

Considering that modem terminals and nodes are realized using computer technology, invention can 
be realized by embedding described algorithms to the device software. Chapter 4.8. of the invention 
essence exposition is also the description of the invention realization. 

7, DESCRIPTION OF INVENTION APPLICATION 

The invention can be applied by building data conununications networks using devices with WTFC 
algorithms. 
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